Information processing method and apparatus

ABSTRACT

An information processing apparatus comprising a touch screen which detects a touch input by a user detects the position and pressure of each of a plurality of pressed points corresponding to a plurality of touch inputs existing at the same time on a touch screen. Then, the front/rear overlap relationship of the plurality of objects displayed on the touch screen is changed based on the difference in pressure between the detected plurality of pressed points.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 12/506,630,filed Jul. 21, 2009, which is based on and claims priority from JapaneseApplication Number JP 2008-196847, filed on Jul. 30, 2008. Thedisclosure of the priority applications, in their entirety, includingthe drawings, claims, and the specifications thereof, are hereinincorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an information processing apparatus andmethod which provide a user interface which uses a touch screen.

2. Description of the Related Art

Recently, there has been a focus on an interface in which objects on ascreen are operated on using hand gestures on a touch screen. Input on atouch screen allows more intuitive operation in comparison with aninterface which uses an input device such as a mouse or keyboard.Furthermore, with recent advances in contact detection devices, it hasbecome possible to identify complicated gestures which allow intuitiveoperation on a touch screen.

In Japanese Patent Laid-Open No. 2001-290585 (hereinafter, patentdocument), a method which detects complicated trajectories of two ormore indicated positions which move at the same time, and performs anoperation on the objects displayed on the touch screen in accordancewith each of the detected trajectories is proposed.

A method which performs an operation on objects on the screen inaccordance with a pressure which presses down on the touch screen isdescribed in the aforementioned patent document. According to thispatent document, an increase in pressure corresponds to a larger amountof page scroll or a larger amount of magnification/reduction, in thecase that the pressure exceeds a constant, the amount of operationbecomes maximal, and when the pressure exceeds a constant, the operationis repeated.

However, while operation by a plurality of pressed forces on the touchscreen is mentioned in the aforementioned patent document, there is nomention of operation designation in accordance with a difference inpressure between a plurality of pressed forces.

SUMMARY OF THE INVENTION

The present invention was made in consideration of such circumstances,and according to an embodiment of the present invention, a userinterface which uses a difference in pressure between a plurality ofpressed forces on a touch screen for greater liberty is provided.

According to one aspect of the present invention, there is provided aninformation processing method in an information processing apparatuscomprising a touch screen which detects a touch input by a user,comprising: detecting position and pressure of each of a plurality ofpressed points corresponding to a plurality of touch inputs existing atthe same time on the touch screen; and changing the front/rear overlaprelationship of a plurality of objects displayed on the touch screenbased on a pressure difference between the plurality of pressed pointsdetected by the detecting step.

Furthermore, according to another aspect of the present invention, thereis provided an information processing apparatus comprising a touchscreen which detects a touch input by a user, comprising: a detectingunit configured to detect position and pressure of each of a pluralityof pressed points corresponding to a plurality of touch inputs existingat the same time on the touch screen; and a changing unit configured tochange the front/rear overlap relationship of a plurality of objectsdisplayed on the touch screen based on a pressure difference between theplurality of pressed points detected by the detecting unit.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a structural example of theinformation processing apparatus according to an embodiment.

FIG. 2 is a diagram representing a structural example of a touch screenof an input device according to an embodiment.

FIG. 3 is a diagram illustrating an example of a data record of a tablewhich stores the pressures of pressed points of the system according tothe embodiment.

FIG. 4 is a diagram illustrating an example of an input operation to theinput device according to the first embodiment.

FIGS. 5A and 5B are diagrams illustrating examples of data records oftables which store trajectories of input positions in the informationprocessing apparatus according to the first embodiment.

FIG. 6 is a diagram illustrating an example of a data record of a tablewhich stores trajectories of input pressures in the informationprocessing apparatus according to the first embodiment.

FIG. 7 is a flowchart illustrating the flow of the processing in theinformation processing apparatus according to the first embodiment.

FIGS. 8A and 8B are diagrams illustrating an example of an objectalignment operation by the information processing apparatus according tothe first embodiment.

FIG. 9 is a flowchart illustrating processing of an object alignmentoperation by the information processing apparatus according to the firstembodiment.

FIGS. 10A and 10B are diagrams illustrating another example of an objectalignment operation by the information processing apparatus according tothe first embodiment.

FIGS. 11A and 11B are diagrams illustrating an example of an objectalignment operation by the information processing apparatus according tothe second embodiment.

FIG. 12 is a flowchart illustrating processing of an object alignmentoperation by the information processing apparatus according to thesecond embodiment.

FIGS. 13A and 13B are diagrams illustrating another example of an objectalignment operation by the information processing apparatus according toa modification of the second embodiment.

FIGS. 14A and 14B are diagrams illustrating another example of an objectalignment operation according to the second embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, preferred embodiments of the present invention will beexplained with reference to the attached drawings.

FIG. 1 is a block diagram roughly illustrating a structural example ofthe hardware of the information processing apparatus 100 according tothe present embodiment.

In FIG. 1, a microprocessor (CPU) 11 controls each structural element ofthe information processing apparatus 100, as well as executes allcalculations and logical determinations, etc. A fixed memory (ROM) 12 isa read only memory which stores control program code for a processingprogram, etc., which is run by the CPU 11. A random access memory (RAM)13 is a writable memory used to temporarily store all data sent from allstructural elements. A storage device 14 is comprised of any type ofdisk device such as a hard disk and/or flash memory, and stores datathat is being processed, image files, and metadata, etc. An input device15 is equipped with a touch screen which executes operation inputthrough contact with the screen. A communication device 16 is comprisedof a line or wireless communication device such as an externalinput/output device such as a USB or a modem, etc., and performs dataexchange with an external device. All of the aforementioned compositionsare connected to each other through an address bus AB, control bus CB,and data bus DB.

The address bus AB is a bus for transferring an address signal fordesignating a structural element targeted for control by the CPU 11. Thecontrol bus CB is a bus for transferring a control signal applied by theCPU 11 to each structural element targeted for control by the CPU 11.The data bus DB is a bus for transferring data between structuralelements.

The information processing unit 100 operates in accordance with eachinput from the input device 15 and with each input provided from thecommunication device 16 through a network. When an input from the inputdevice 15 or an input from the communication device 16 is provided, aninterrupt signal is sent to the CPU 11. The CPU 11 reads out eachcontrol signal stored in the storage device 14 in accordance with theinterrupt signal, and executes each control according to those controlsignals.

FIG. 2 is a diagram illustrating a touch screen 200 equipped by theinput device 15 in the information processing apparatus 100 of thepresent embodiment. The touch screen 200 has a touch input unit 201which inputs operations from the operator, and a display unit 202 whichdisplays an image. Furthermore, the touch screen comprises a pressedposition detection unit 203 which detects each position (coordinate) ofa plurality of pressed points corresponding to a plurality of touchinputs on the screen, and a pressure detection unit 204 which detectseach pressure of a plurality of pressed points. An example in which thescreen is pressed by a finger on the left and right hands at the same isillustrated in FIG. 2.

In the touch screen 200, from the lower left of the screen, the X-axisis taken in the rightward direction, while the Y-axis is taken in theupward direction, and the coordinate positions and pressures of aplurality of points pressed on the screen are detected. In FIG. 2, asituation in which a coordinate of a point pressed by a finger on theleft hand at time t is detected to be At(x,y), while a coordinate of apoint pressed by a finger on the right hand is detected to be Bt(x,y) isillustrated. Furthermore, at the pressed points of the left and rightfingers, a shadow with an area which becomes larger as the pressurebecomes larger is displayed, and the user can be aware of the size ofthe pressures (difference) of each pressed point. That is, on the touchscreen 200 of the present embodiment, a display shape is changed inaccordance with the pressure to display a pressed point such that theuser can be aware of the pressures (or the difference in pressure) of aplurality of detected pressed points.

FIG. 3 is a data table illustrating the contact pressure of each pressedposition detected by the touch screen 200 when it is pressed asillustrated in FIG. 2. The CPU 11 records the pressures of the pluralityof pressed points at time t detected by the touch screen 200 (pressedposition detection unit 203 and pressure detection unit 204) as a table300 in, for example, a RAM 13.

FIG. 4, FIG. 5A, FIG. 5B, and FIG. 6 are diagrams explaining processingwhich traces trajectories in the case that pressed points are slid overthe touch screen.

FIG. 4 illustrates the pressed points at positions At1 and Bt1 at timet1 being slid over the screen to At5 and Bt5, respectively, at time t5over times t2, t3 and t4. Furthermore, at this time, as shown in FIG. 5Aand FIG. 5B, the CPU 11 records the positional coordinate of each point(positional coordinate of each point as the trajectories of each ofpoint A and point B at reference times t1, t2, t3, t4, and t5) detectedby the pressed position detection unit 203 in, for example, the RAM 13.At the same time, as shown in FIG. 6, the CPU 11 records the pressure ofeach point at each positional coordinate detected by the pressuredetection unit 204 in, for example, the RAM 13.

In this manner, the trajectories of a plurality of an operator's fingerson the screen are recorded from contact to release. In the presentembodiment, the operation of an object on the screen is interpreted inaccordance with the thus obtained trajectories of the positions andpressures of the pressed points on the touch screen. Hereinafter,processing which interprets an operation to be executed from theobtained trajectories of the positions and pressures of two or morepressed points will be specifically explained.

FIG. 7 is a flowchart illustrating the flow of the processing of thepresent embodiment. Moreover, the processing illustrated in FIG. 7 isrealized by a CPU 11 executing a program stored in a ROM 12 or a programloaded into a RAM 13.

In step S701, the CPU 11 determines whether or not a pressed input wasdetected by the touch screen 200. If a pressed input has not beendetected, detection is awaited. In the case that a pressed input hasbeen detected, in step S702, the CPU 11 determines whether or not anending operation has been executed. In the present embodiment, an endingoperation is determined to have been executed in the case that apressure on the touch input unit 201 of the touch screen 200 isreleased. In the case that it is determined that an ending operation hasbeen executed, the CPU 11 ends the present processing.

When it is determined that an ending operation has not been executed instep S702, the process advances to step S703. In step S703, the CPU 11interprets which operation is designated by the input on the touchscreen 200 based on the pressed position and pressure detected on thetouch screen 200. Then, in step S704, the CPU 11 executes the operationinterpreted in step S703. In the aforementioned manner, in the case thatan ending operation has not been executed, the next input is awaited,and when an input is detected, processing which interprets and executesthat input is repeated.

FIGS. 8A and 8B are diagrams explaining an embodiment of processingwhich aligns objects on the screen based on the trajectories of thepositions and the trajectories of the pressures of two pressed points.

At time t1, a collection of objects 801 is displayed on the touch screen200 as shown in FIG. 8A, and this collection of objects 801 has beenselected by a predetermined operation on the touch screen 200(hereinafter described step S901). Then, two points (At1 and Bt1) inthis collection of objects 801 are pressed as shown in FIG. 8A. Fromthis state, the two pressed points A and B are slid in directions suchthat they become further away from each other until time tn, and thestate in which point A has moved to coordinate Atn and point B has movedto coordinate Btn is shown in FIG. 8B.

By this operation, each object in the group of objects 801 on the screenis aligned between point A and point B as the object array 802.Furthermore, alignment is done such that objects in the direction ofpoint B for which the pressure is smaller are displayed in front basedon the difference in pressure between point A and point B. For thisreason, by displaying objects which are pressed harder by a finger suchthat they sink lower (deeper into the screen), an intuitive operationenvironment can be provided to the user.

Using FIG. 9, the processing of the CPU 11 which realizes theaforementioned alignment operation will be explained. FIG. 9 is aflowchart corresponding to input interpretation processing in step S703in FIG. 7 and operation execution processing in step S704.

In step S901, the CPU 11 sets the plurality of objects displayed on thescreen to a selected state by a predetermined operation input on thetouch screen 200. For example, objects inside a circle are in a selectedstate by a gesture called a loop operation in which the perimeter of acollection of objects to be selected is enclosed by a circular shape (ora closed figure) made by a finger. Of course, the selection method isnot limited to this operation. For example, by designating one object,an object group for which there is at least one overlapping portionincluding the designated object may be selected. Thus, in step S901, aplurality of objects is selected from the objects displayed on the touchscreen 200 in accordance with a touch input on the touch screen 200.

In step S902, the CPU 11 determines whether or not two pressed pointsare detected (exist) at the same time on the touch screen 200 (thepoints to not have to be pressed at the same time). In the case thatthey are detected, the process advances to step S903. In the case thatsuch pressed points have not been detected after a predetermined amountof time passes, or if a pressed point with an intention for a differentoperation is detected, the present operation is ended. As a pressedpoint with an intention for a different operation, a case in which onlyone pressed point exists and it begins to move, etc., can be given.

In step S903, the CPU 11 determines whether or not both of the twopressed points detected in step S902 are pressing the collection ofobjects selected in step S901 at the same time. The CPU 11 executes theaforementioned determination by comparing the positional coordinates(the region in which the collection of objects exists) of the collectionof objects with the positional coordinates of the pressed points, andchecking whether or not there is an overlap between the two. In the casethat there are two pressed points on the selected collection of objectssuch as At1 and Bt1 in FIG. 8, the process advances to the next stepS904. Otherwise, it is determined that there is another operation, andthe present operation is ended.

In step S904, the CPU 11 interprets the operator's operation to beexecuted to be an operation to align the selected objects, since the twopressed points are on the selected objects at the same time by thedetermination in step S903.

In step S905, the CPU 11 detects whether or not the relative distancebetween the two pressed points is changing. In the case that therelative distance between the two pressed points is not changing, nooperation is performed. In the case that a change in the relativedistance is detected, the process moves to the next step S906.

In step S906, the CPU 11 aligns the objects between the two pressedpoints after movement based on the alignment designation interpreted instep S904. That is, the CPU 11 aligns each object in the designatedcollection of objects 801 such that the two pressed points Atn and Btnafter movement are the ends, and the centers of the objects are equallyspaced apart, as shown in the object array 802 in FIG. 8B. Moreover, inthe present embodiment, although the relative direction of movement ofthe pressed points is a straight line, and the objects are arranged on astraight line with endpoints at Atn and Btn, the alignment method is notlimited to this. For example, the objects may be aligned on a circulararc as in fanned playing cards by sliding the two pressed points in theshape of a circular arc (refer to FIG. 11A). In this case, a circlewhich fits the arc-shaped trajectory of the pressed points iscalculated, and the objects are arranged on a circular arc with thecoordinates Atn and Btn on the calculated circle as endpoints. In theaforementioned manner, in the case that the two pressed points detectedin the display region of a plurality of objects selected in step S901,the plurality of selected objects are arranged on a straight line orcurve connecting the two pressed points after movement such that theobjects are equally spaced apart.

In step S907, the CPU 11 determines whether or not an overlap betweenaligned and adjacent objects is detected. If the distance between Atnand Btn is small, adjacent objects will be aligned so as to overlap, asshown in the object array 802 in FIG. 8B. In the case that an overlap isdetected, the process advances to step S908. If it is not detected, thepresent operation is ended.

In step S908, the difference in pressure between the two pressed pointsis calculated. The difference in pressure between the two pressed pointsis calculated by, for example, comparing the average pressure over thetrajectories of the two pressed points in FIG. 6. In FIG. 6, thepressure of each point is constant over the entire time, and the averagepressure of point A is 150, while the average pressure of point B is100, and the pressure of point A is 50 larger than the pressure of pointB. However, the pressures compared here are not limited to the averageover the trajectory. The pressures at the starting point at time t1, orthe pressures at the ending point at time t5 may also be compared.

In step S909, the CPU 11 aligns the direction of overlap such thatobjects toward the side for which the pressure calculated in step S908is smaller are displayed in front. In the object array 802 in FIG. 8B, acase in which the pressure of point B is the smaller of the pressures ofpoint A and point B. That is, after alignment, the object array 802 isaligned such that images toward the side of Btn for which the pressureis smaller are displayed in front of images toward the side of Atn.

In the aforementioned manner, in the present embodiment, in the casethat a plurality of objects randomly arranged on a touch screen 200 areselected, that plurality of selected objects are arranged to be lined upaccording to a predetermined sorting standard on a straight line orcurve connecting two pressed points after movement. Here, the front/rearoverlapping relationship of the plurality of objects arranged betweenthose two pressed points changes based on the detected difference inpressure among the plurality of pressed points. That is, the directionof overlap of the plurality of objects is determined based on thedifference in pressure between the two pressed points. Morespecifically, the overlap of the objects is changed such that theobjects are displayed in front from the pressed point for which thepressure is larger toward the pressed point for which the pressure issmaller, and intuitive operation becomes possible. Moreover, as thesorting standard, an object file name order, an object time stamp order,etc., can be given.

FIGS. 10A and 10B are diagrams illustrating an operation on alignedobjects, in contrast to FIGS. 8A and 8B, which illustrate an example ofaligning unaligned objects. That is, in the case that an object array ofa plurality of aligned objects is selected, that plurality of objectsare arranged on a straight line or curve connecting two pressed pointsafter movement while maintaining the alignment order of the objectarray. Hereinafter, although a case of an operation on an object arrayin which the objects are arranged without overlapping is explained, itis clear that it is possible to execute similar processing for anoperation on an object array in which a portion of the objects overlap.

At time t1, as shown in FIG. 10A, an object array 1001 which isregularly aligned without any overlap between objects is displayed.Moreover, the object array 1001 targeted for operation is selected bythe aforementioned processing in step S901.

In step S903, in the case that it is determined that there are twopressed points on the aligned objects, the objects on the trajectory ofa straight line connecting the two pressed points are put in a selectedstate. Moreover, the aligned objects are the object array obtained byaligning the collection of objects 801 as in FIG. 8B by an alignmentoperation, as shown in FIGS. 8A and 8B. Alternatively, as shown in FIG.10A, in addition to the objects selected by the pressed points At1 andBt1, two objects on a straight line connecting At1 and Bt1 may beselected as objects belonging to the object array 1001, that is, alignedobjects. Moreover, “the pressed points are on the aligned objects”refers to pressed points detected in the region of any one or moreobjects among the selected aligned objects.

As shown in FIG. 10B, over time t1 to tn, the point A is slid fromcoordinate At1 to Atn, while the point B is slid from coordinate Bt1 toBtn. The CPU 11 then aligns the selected objects between Atn and Btn instep S906 in FIG. 9. Here, when the distance between Atn and Btn isbelow a constant value, the CPU 11 detects an object overlap (step S907in FIG. 9). Then, the CPU 11 overlaps and aligns the objects such thatobjects on the side with the lower pressure are in front, and displaysthem (step S909).

Moreover, in contrast to FIGS. 8A and 8B, FIGS. 10A and 10B illustrate acase in which the pressure of point B is larger than the pressure ofpoint A. Therefore, by the processing in step S909, objects closer toAtn are displayed so as to be in front, as shown in the object array1002. Moreover, in the example of FIG. 10B, the objects are aligned suchthat the centers are aligned with each of the pressed points A and Bafter movement. That is, point A and point B are object centers, and theobjects are aligned such that each object is arranged to be equallyspaced apart in the horizontal and vertical directions, respectively.Moreover, although the pressing points are the object centers in thisexample, there is no limitation to this, and objects may be alignedwithout relation to the positions of the pressing points on the objects.For example, the interval of the objects may be changed in only thehorizontal direction, as shown in FIG. 10A.

As mentioned above, according to the first embodiment, by introducing analignment method based on the difference in pressure between a pluralityof pressed points, the operation which was conventionally executed intwo steps of aligning the objects and then changing the overlapdirection can be executed in one step.

FIGS. 11A and 11B are diagrams illustrating an example of an objectoperation according to the second embodiment.

At time t1, as shown in the object array 1101 in FIG. 11A, the pluralityof objects is aligned in an overlapping fan shape. Furthermore, in theobject array 1101, the objects are regularly aligned such that those onthe right side are in front. Furthermore, at time t1, the points At1 andBt1 are pressed on the aligned objects at the same time, and thepressures of the pressed points A and B are such that point A<point B.In this case, the CPU 11 interprets the pressed operation to be adesignation to change the overlap direction of the aligned objects.Next, the overlap of the objects in the object array is changed suchthat objects toward the pressed point for which the pressure is smallerare displayed in front, as shown in the object array 1102 in FIG. 11B.

FIG. 12 is a flowchart explaining alignment operation processingaccording to the second embodiment. Moreover, FIG. 12 corresponds tostep S703 (input interpretation processing) and step S704 (operationexecution processing) in FIG. 7.

In step S1201, the CPU 11 determines whether or not two pressed pointsare detected at the same time on the touch screen 200. If they aredetected, the process advances to step S1202. In the case that suchpressed points have not been detected after a predetermined amount oftime passes, or in the case that a pressed point with an intention for adifferent operation is detected, the present operation is ended. As apressed point with an intention for a different operation, a case inwhich only one pressed point exists and it begins to move, etc., can begiven.

In step S1202, the CPU 11 determines whether or not the two pressedpoints detected in step S1201 are on the same aligned object. In thecase that it is determined that they are on the same aligned object, theprocess advances to step S1203. Otherwise, the process is ended.Moreover, when “determining whether or not the two pressed pointsdetected in step S1201 are on the same aligned object”, whether or notthey are on the same aligned object is determined in the followingmanner. (1) The aligned objects are selected by a loop operation(however, in that case, step S1203 is unnecessary). (2) Because thealignment operation in FIG. 9 is definitely executed on the alignedobjects beforehand, information that the plurality of objects belongs tothe same aligned objects (for example, a group ID) is stored in the RAM13 during the alignment operation. In this case, when the single objectsunder each of the two pressed points have the same group ID, it isdetermined that the operation is directed to the aligned objects as awhole.

In step S1203, the CPU 11 selects all objects included in the alignedobjects. For example, in the case that the positions At1 and Bt1 of theobjects on both ends of the aligned objects aligned in a fan shape inFIG. 11A are pressed at the same time, all objects comprising thealigned objects (object array 1101) are put in a selected state.

In step S1204, the CPU 11 determines whether or not there are objects inthe aligned objects selected in step S1203 that overlap each other. Inthe case that an overlap is detected between objects, the processadvances to step S1205. If one is not detected, the present processends.

In step S1205, the CPU 11 calculates the difference in pressure betweenthe two pressed points. For example, the CPU 11 calculates thedifference in pressure from the pressure of the two points recorded asin FIG. 6.

In step S1206, the CPU 11 determines whether or not the difference inpressure calculated in step S1205 exceeds a predetermined thresholdvalue (pressure difference threshold value). This is processing toexecute an operation only in the case that the operator intentionallychanges the pressure of the two pressed points, since strictly speaking,a difference in pressure between two points will emerge even when theoperator intends to press the two points with the same strength.Moreover, the aforementioned threshold value may be set by the user. Inthe case that it is determined that the difference in pressure betweenthe two detected points exceeds a threshold value, the process advancesto step S1207. On the other hand, if it is not exceeded, the presentprocess is ended.

In step S1207, the CPU 11 determines whether or not the pressures of thetwo pressed points on the aligned objects continue for more than apredetermined amount of time while maintaining the difference inpressure (difference in pressure which exceeds the threshold value)detected in step S1206. In the case that the aforementioned differencein pressure is maintained for more than a predetermined amount of time,the process advances to step S1208, and in other cases the process isended.

In step S1208, the CPU 11 changes the display mode of the alignedobjects such that objects on the side on which the pressure is smallerare displayed in front. In FIG. 11A, pressed points for which thedifference in pressure is such that point A<point B over time t1 to tnare detected on the objects aligned and displayed such that the objectson the right side are displayed in front. Here, in the case that thetime tn-t1 exceeds a predetermined time threshold value and thedifference in pressure between point A and point B exceeds apredetermined pressure difference threshold value, the CPU 11 interpretsthese pressed points to be an operation which changes the overlapdirection of the aligned objects. Next, the CPU 11 changes the overlapdirection of the aligned objects such that objects toward the side onwhich the pressure is smaller, that is, closer to point A, are moved tothe front, as shown in the object array 1102 in FIG. 11B.

In the aforementioned manner, in the second embodiment, in the case thattwo pressed points are detected on an object array in which a pluralityof objects is aligned with overlap, the direction of overlap of theobjects comprising the object array is changed in accordance with thedirection of the difference in pressure between those two pressedpoints. Moreover, in the aforementioned explanation, although pressedpoints on objects at both ends of the object array are indicated, thereis no limitation to this. The overlap of the objects may be changed inaccordance with a difference in pressure between two pressed pointsdesignated within the display range of the object array or within arange including the proximity of the object array. For example, as shownin FIG. 14A, in the case that At1 and Bt1 are designated and these areheld until time tn, the direction of overlap is changed as shown in FIG.14B. That is, the direction of overlap of the object array is changed inaccordance with the pressure difference direction 1401, and the sameresult as in FIG. 11B is obtained.

Thus, it is possible to change the direction of overlap of all thealigned objects in 1 step by using the difference in pressure between 2pressed points.

FIGS. 13A and 13B are diagrams illustrating a modification of the secondembodiment.

As shown in FIG. 13A at time t1, there is a collection of objects 1301having overlap between the objects, and At1 and Bt1 are pressed at thesame time at time t1. Here, the pressures of the pressed points A and Bare such that point A<point B. In this case, the CPU 11 interprets thepressed input as an instruction to change the direction of overlap ofthe objects in the collection of objects 1301 pressed by point A andpoint B. The CPU 11 then changes the direction of overlap such that theobject designated by the pressed point with the smaller pressure isdisplayed in front for only the objects pressed by point A and point B,as shown in the collection of objects 1302 in FIG. 13B.

In the above manner, in the aforementioned modification, in the casethat two objects which partially overlap each other are selected fromamong a plurality of objects by two pressed points, the front/reardisplay relationship of both objects is changed based on the differencein pressure between the two pressed points. More specifically, of thetwo objects selected by the two pressed points, the object correspondingto the pressed point with the smaller pressure is displayed in front.According to such control, because it is possible to change thedirection of overlap for any arbitrary objects in a collection ofobjects in one step by using the difference in pressure between twopressed points, operability can be improved.

According to each embodiment of the present invention exemplified above,it is possible to provide an environment which allows intuitiveoperation of objects on a screen by using a difference in pressure whichselects a plurality of objects by pressing on a touch screen.

Although embodiments were described in detail above, the presentinvention can be embodied as, for example, a system, device, method,program, or storage medium, etc. Specifically, it may be applied in asystem comprising a plurality of devices or an apparatus comprised of asingle device.

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment(s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

What is claimed is:
 1. An information processing apparatus having atouch screen which detects a touch input, comprising: an obtaining unitconfigured to obtain position of each of a plurality of pressed pointscorresponding to a plurality of touch inputs existing at the same timeon the touch screen; a display unit configured to display a plurality ofobjects on the touch screen; and a display changing unit configure to,if a movement of positions of the plurality of pressed points frompositions of the plurality of objects displayed by the display unit isdetected, change locations of the plurality of objects based on themovement.
 2. The apparatus according to claim 1, wherein the displaychanging unit changes a display such that the plurality of objects arealigned.
 3. The apparatus according to claim 1, wherein the displaychanging unit changes a display interval of the plurality of objects. 4.The apparatus according to claim 1, wherein the display unit displaysthe plurality of objects overlapping each other, and the displaychanging unit changes a display of the plurality of objects such thatthe plurality of objects spread.
 5. The apparatus according to claim 1,wherein the display changing unit changes front/rear overlaprelationship o the plurality of objects.
 6. The apparatus according toclaim 1, wherein the display changing unit detects, as the movement,that positions of the plurality of pressed points move such that theybecome further away from each other.
 7. The apparatus according to claim1, wherein the obtaining unit further obtains pressure of each of theplurality of pressed points.
 8. The apparatus according to claim 7,wherein the display changing unit changes display of the plurality ofobjects based on difference in pressure obtained by the obtaining unitbetween the plurality of pressed points.
 9. An information processingapparatus having a touch screen which detects a touch input, comprising:an obtaining unit configured to obtain pressure of each of a pluralityof pressed points corresponding to a plurality of touch inputs existingat the same time on the touch screen; and a generation unit configuredto generate an operation instruction based on pressure of the pluralityof pressed points.
 10. The apparatus according to claim 9, wherein theobtaining unit further obtains a relationship between pressures of theplurality of pressed points, and the generation unit generates operationinstruction based on the relationship of the pressure.
 11. The apparatusaccording to claim 9, wherein the obtaining unit further obtainsmovement of the plurality of pressed points, and the generation unitgenerates operation instruction based on pressure of the plurality ofpressed points and the movement.
 12. The apparatus according to claim 9,further comprising a display control unit configured to perform displaycontrol for one or a plurality of objects displayed on the touch screenbased on the pressure of the plurality of pressed points, which havebeen obtained by the obtaining unit.
 13. The apparatus according toclaim 12, wherein the display control unit aligns the plurality ofobjects based on pressure of the plurality of pressed points.
 14. Theapparatus according to claim 12, wherein the plurality of object have afront/rear overlap relationship, and the display control unit changesthe front/rear overlap relationship of the plurality of objects based onthe pressure.
 15. An information processing apparatus having a touchscreen which detects a touch input, comprising: an obtaining unitconfigured to obtain pressure of each of a plurality of pressed pointscorresponding to a plurality of touch inputs existing at the same timeon the touch screen; a display control unit configured to controldisplaying, on the touch screen, a representation corresponding to thepressure obtained by the obtaining unit, such that a user can recognizepressure of the plurality of pressed points.
 16. The apparatus accordingto claim 13, wherein the obtaining unit further obtains location of aplurality of pressed points, and the display control unit displays anobject at a position corresponding to the location, which has a sizecorresponding to the pressure.
 17. A control method of an informationprocessing apparatus having a touch screen which detects a touch inputby a user, comprising: an obtaining step of obtaining position of eachof a plurality of pressed points corresponding to a plurality of touchinputs existing at the same time on the touch screen; a display step ofdisplaying a plurality of objects on the touch screen; and a displaychanging step of changing, if positions of the plurality of pressedpoints obtained in the obtaining step move from positions of theplurality of objects displayed in the display step, locations of theplurality of objects based on the movement.
 18. A control method of aninformation processing apparatus having a touch screen which detects atouch input, comprising: an obtaining step of obtaining pressure of eachof a plurality of pressed points corresponding to a plurality of touchinputs existing at the same time on the touch screen; a generation stepof generating an operation instruction based on pressure of theplurality of pressed points.
 19. A control method of an informationprocessing apparatus having a touch screen which detects a touch input,comprising: an obtaining step of obtaining pressure of each of aplurality of pressed points corresponding to a plurality of touch inputsexisting at the same time on the touch screen; a display control step ofcontrolling to display, on the touch screen, a representationcorresponding to the pressure obtained in the obtaining step, such thata user can recognize pressure of the plurality of pressed points.
 20. Anon-transitory computer readable storage medium storing a program forcausing a computer to execute a method of an information processingapparatus having a touch screen which detects a touch input by a user,the method comprising: an obtaining step of obtaining position of eachof a plurality of pressed points corresponding to a plurality of touchinputs existing at the same time on the touch screen; a display step ofdisplaying a plurality of objects on the touch screen; and a displaychanging step of changing, if positions of the plurality of pressedpoints obtained in the obtaining step move from positions of theplurality of objects displayed in the display step, locations of theplurality of objects based on the movement.
 21. A non-transitorycomputer readable storage medium storing a program for causing acomputer to execute a method of an information processing apparatushaving a touch screen which detects a touch input, the methodcomprising: an obtaining step of obtaining pressure of each of aplurality of pressed points corresponding to a plurality of touch inputsexisting at the same time on the touch screen; a generation step ofgenerating an operation instruction based on pressure of the pluralityof pressed points.
 22. A non-transitory computer readable storage mediumstoring a program for causing a computer to execute a method of aninformation processing apparatus having a touch screen which detects atouch input, the method comprising: an obtaining step of obtainingpressure of each of a plurality of pressed points corresponding to aplurality of touch inputs existing at the same time on the touch screen;a display control step of controlling to display, on the touch screen, arepresentation corresponding to the pressure obtained in the obtainingstep, such that a user can recognize pressure of the plurality ofpressed points.